TPIFile is a sample TPI module that lets you open a “TPIFile” endpoint, connect that endpoint to a Mac OS file using an FSSpec, and then read that file using standard Open Transport API calls (eg OTRcv).
Theory of Operation
The sample has three components:
1. TPIFile — An OT module that actually responds to open, connect, and read requests.
2. TPIFileRegister — An application that registers a port called “TPIFile” so that the client can go OTOpenEndpoint(OTCreateConfiguration("TPIFile"), ...) and successfully open the module.
3. TPIFileTest — An application that opens a "TPIFile" endpoint, connects that endpoint to a file using a hardwired file name, reads the data out of the file, and prints it to a console window.
TPIFile uses asynchronous File Manager calls throughout. STREAMS modules are not allowed to “block”. OT interprets this as meaning they are not allowed to make synchronous File or Device Manager calls.
Important: Making File Manager calls from a STREAMS module works on the current implementation of OT on the traditional Mac OS, but it is not recommended. If you find that your OT module requires access to the File Manager, chances are that you need to restructure your module into client-side and kernel-side parts that communication via streams.
Important: The purpose of this sample is to provide a useful example of a STREAMS module without requiring access to hardware, not to provide production quality access to the file system. For example, TPIFile does not currently work if the file is resident on a network volume. See the comment in TerminateStreamModule in “TPIFile.c” for an explanation of why. In summary, this is an artifact of the previous warning.
Packing List
The TPIFile package contains the following items:
• Read Me About TPIFile — This document.
• TPIFile.mcp — A PPC project file with targets to build the “TPIFile” module, the TPIFileRegister application, and the TPIFileTest application.
• TPIFile.c — Source to the TPIFile module.
• TPIFileRegister.c — Source to the TPIFileRegister application.
• TPIFileTest.c — Source to the TPIFileTest application.
• TPIFile.h — Public declarations for the TPIFile module. These are shared between the module itself and the various helper programs, ie TPIFileRegister and TPIFileTest.
• TPIFile — A compiled version of the TPIFile module.
• TPIFileRegister — A compiled version of the TPIFileRegister application.
• TPIFileTest — A compiled version of the TPIFileTest application.
• Instrumentation Stuff — Stuff needed to build the instrumented version of the TPIFile module (the TPIFile.Inst target in the project). See the Instrumentation for Fun and Profit section of this document for more details about this effort.
Installation and Execution
To test the sample, do the following:
1. Drag the “TPIFile” module into your Extensions folder.
2. Run the “TPIFileRegister” application.
3. Run the “TPIFileTest” application, choose test "a", and see how it displays the contents of the “Victim” file to the screen.
Building the Sample
This sample was built using the Metrowerks CodeWarrior Pro 2 environment with Universal Interfaces 3.3.1. It should build just fine with more modern versions of CodeWarrior.
To rebuild the sample, open the project, click on the Targets tab, select the TPIFileTest, TPIFileRegister, and TPIFile targets, and then choose Make from the Project menu.
Restrictions
TPIFile has a number of limitations that are unlikely to be removed. These ‘working as designed’ restrictions make the code simpler:
1. TPIFile does not support writing data to the file. Writing data would complicate the sample because it would require file mark management, and pose file creation questions.
2. TPIFile does not support passive endpoints, ie creating an endpoint and waiting for a connection. It most probably doesn't make any sense to do so.
3. TPIFile does not currently work if the file is resident on a network volume. See the comment in TerminateStreamModule in “TPIFile.c” for an explanation of why. I could have worked around this but the workaround would have been really ugly, and very much counter to the spirity of sample code.
See also the Caveats section of this document for a description of some of the problems that I would like to fix in the future.
State Transitions
TPI modules are expected to handle messages in a very specific fashion, as defined by the state transition diagrams in the TPI specification. Because TPIFile only supports a subset of the functionality embodied in TPI module, it only needs to deal with a subset of these state transitions.
For more information about TPI states and transitions, check out the TPI Specification available from the OT web page:
<http://developer.apple.com/macos/opentransport/>
TPIFile deals with the following state transitions:
sta_0 (unbnd) + bind_req -> sta_1 (w_ack_b_req)
sta_1 (w_ack_b_req) + bind_ack -> sta_3 (idle)
sta_1 (w_ack_b_req) + error_ack -> sta_0 (unbnd)
The handling of these transitions is all bundled in DoBindRequest. TPIFile has nothing particularly exciting to do in response to a bind, so it can handle the bind request synchronously. This means that it can detect success or failure immediately, and doesn’t need the intermediate w_ack_b_req state.
sta_3 (idle) + unbind_req -> sta_2 (w_ack_u_req)
sta_2 (w_ack_u_req) + ok_ack1 -> sta_0 (unbnd)
sta_2 (w_ack_u_req) + error_ack -> sta_3 (idle)
The handling of these transitions is all bundled in DoUnbindRequest. Again, TPIFile handles unbinds synchronously, so there is no intermediate w_ack_u_req state.
sta_3 (idle) + conn_req -> sta_5 (w_ack_c_req)
sta_5 (w_ack_c_req) + error_ack -> sta_3 (idle) -- A
sta_5 (w_ack_c_req) + ok_ack1 -> sta_6 (w_con_c_req) -- B
TPIFile deals with connection requests in DoConnectRequest. Part of the connection request, namely the checking of the parameter, is handled synchronously. If the parameter checking fails, DoConnectRequest will immediately go through state transition A. If the parameter check succeeds, DoConnectRequest starts to open the file (using PBHOpenAsync) and sends up the OK ACK message (transition B above).
sta_6 (w_con_c_req) + conn_con -> sta_9 (data_t)
sta_6 (w_con_c_req) + discon_ind1 -> sta_3 (idle)
TPIFile deals with this state transition in DoConnectConfirm. If the PBHOpenAsync succeeds, DoConnectConfirm will generate a connect confirmation and proceed to the data transfer state. If it fails, we send up a disconnection indication to let the client know of the failure.
This is the final state transition in DoConnectConfirm. If we received a disconnect request message while the PBHOpenAsync was in progress, we remember that we got that message by setting the state to w_ack_dreq6. When the PBHOpenAsync completes, DoDisconnectRequestAck executes, notices this state and sends the confirmation of the disconnection request.
sta_9 (data_t) + data_ind -> sta_9 (data_t)
This is the most obvious state transition. Sending data upstream (as done in DoDataIndication) does not cause a change of state.
sta_9 (data_t) + discon_ind1 -> sta_3 (idle)
This is a slightly more subtle variant of the above. If the PBReadAsync failed with an error, we generate a disconnection indication and proceed to the idle state.
If we're transferring data and we get a disconnect request, we flag the request by going into state w_ack_dreq9. When DoDisconnectRequestAck runs, it checks for this state and sends up the acknowledgement.
Important: One state transition that is not covered in TPI is related to closing a stream. OT can call your module’s close routine while the endpoint is in virtually any state. You are expected to deal with this. Handling these out of state close operations proved to be one of the hardest parts of TPIFile.
Caveats
• A significant number of code paths in TPIFile have never been tested. Doing good testing on TPI modules is very tricky, and in a lot of cases I haven’t had time to verify that the code works as designed. I did my best given the limited time I had. Sorry.
• TPIFile does not deal with out of memory conditions well. At the moment the sample just trips an OTAssert if it ever runs out of memory. The correct strategy is to use mi_bufcall to schedule a queue to be called when memory is available again. If I had time to fix one thing in this sample, that would be it.
• "TPIFileTest.c" is hardwired to read a file called “Victim” in the same folder as the TPIFileTest application. Working around this would be a no-brainer, but having the filename hardwired actually helps when you’re testing.
• There is no 68K version. This would require me to mess with ASLM. While I can fly ASLM, I figure it’s better for me to spend my time writing sample code as opposed to sample build systems.
Instrumentation for Fun and Profit
One of the problems with writing Open Transport modules is that they are a pain to debug. While it is possible to use a variety of debuggers on an OT module (eg MacsBug, Power Mac Debugger, and so on), they all suffer similar problems:
• You can only trace one routine invokation at a time, so it’s very hard to get an overview of your entire module’s behaviour. While you can set breakpoints on the various open, close, put, service and interrupt routines, it is painful to trace the execution of the module from there.
• Tracing a routine in the debugger will cause a significant variation in the timing of your module, which has the potential to hide bugs.
The classic workaround for debugging networking modules has been to implement a “log module”, a fast set of routines for recording “trace points” in RAM and dumping them out again at some non-critical time. In fact, Open Transport implements at least two log/trace mechanisms internally.
When I came to debug TPIFile I came across the two problems given above and wanted to include a logging mechanism. Rather than invent my own, however, I decided to use one already developer at Apple and available to developers. The Instrumentation SDK is a cool environment for capturing and analysing trace logs (and much more besides). You can download the full SDK from the Apple developer web site.
<http://developer.apple.com/sdk/>
When I implemented the Instrumentation SDK into the TPIFile project I was careful to avoid introducing any hard dependencies on it. You can compile TPIFile with or without instrumentation. In fact, the project contains two targets that do just that. In addition, the files that are only relevant to the instrumented build are buried in their own folder. The “Instrumentation Stuff” folder contains the following items.
• TPIFile.Inst.prefix — A special prefix file used by the instrumented target to set the INSTRUMENTATION_ACTIVE compiler variable.
• Instrumentation SDK 1.0.5 Bits — The parts of the Instrumentation SDK needed to build the instrumented target.
• Read Me About Tests — A read me file describing the following two items.
• Async I/O Tests — See “Read Me About Tests”.
• Close Tests — See “Read Me About Tests”.
• Latest Instrumentation SDK — A URL to the Apple developer web site, where you can pick up any updates to the Instrumentation SDK.
To build an instrumented version of the module, open the project, click on the Targets tab, select the TPIFileTest.Inst, TPIFileRegister, and TPIFile targets, and then choose Make from the Project menu.
See the documentation that came with the SDK for information on capturing and interpreting instrumentation information.
Credits and Version History
If you find any problems with this sample, mail <dts@apple.com> and I’ll try to fix them up.
Version 1.0d1 (Mar 1997) was a pre-release of this sample that I distributed to a limited number of developers.
Version 1.0b1 (Apr 1997) was the first broadbased release. It added the “TPIFile.h” interface file, and changed the address type to use an FSSpec rather than a full path. Added support for closing a stream while there’s an asynchronous I/O operation in progress. Fleshed out the state transition coverage. Added M_FLUSH request processing. Got rid of most of the “return in the middle of a routine” constructs (bad idea anyway) in preparation for instrumentation. Added instrumentation, and used it to track down some nasty problems. Engaged real asynchronous File Manager operations by turning on the noCacheBit. Did a whole bunch of testing.
Version 1.0.1b1 (May 2000) is an update to compile with the latest tools and interfaces. Also updated the documentation to account for 2 years of history.
